Fujitsu Laboratories TREC8 Report - Ad hoc, Small Web, and Large Web Track

This year a Fujitsu Laboratory team participated in three tracks:that is ad hoc, small web track, and large web track. As basic techiniques, we compared four popular stemmers, and we made simple removing stop pattern techniques for TREC queries. For the ad hoc task, and small web track, we used the same techiniques. We experimented with area weighting, co-occurence boosting, bi-gram utlization, and reranking by bi-gram extraction from pilot search. The e ect of blind application with those techiniques is rather limited, or even uncertain in the TREC8 experiment. What we can say from TREC8 result is that blind application of co-occurence boosting and area weighting may be e ective for the small web track. They requerie query dependent application. In the large web track, our main interest is efciency, that is how much resources are required to process 100GB of web text and 10000 real web queries in practical time. Using a statistical based language type checker, we can eliminate 23% of nonEnglish text. This leads to speeding up a indexing and reducing the index size. The search speed for an inverted le is CPU intensive if the target machine has main memory in excess of 10-25% of the index size. So with simple, but e ective index compression methods, the throughput of query processing is about 0.54-1.1 query/second even by a single 300MHz Ultra-sparc processor. 1 System Description